Chapter 16 Probability (Concepts)

Random Experiments and Sample Spaces

Probability is the mathematical study of chance and uncertainty. It provides tools to quantify the likelihood of different outcomes occurring in situations where the result is not predictable. The fundamental concepts in probability theory are those of a random experiment and its sample space.

To understand probability, it is essential to first be clear about the precise meaning of the terms used. The entire theory is built upon the distinction between processes that are predictable and those that involve an element of chance.

1. Experiment

In the context of science and mathematics, an Experiment is a well-defined procedure or action that can be repeated and results in a set of observable outcomes. It is a general term for any process that generates data or observations.

Experiments can be broadly classified into two categories: deterministic and random.

2. Deterministic Experiment

A Deterministic Experiment is an experiment whose outcome is certain and predictable if it is performed under identical conditions. There is no element of chance involved. The result is uniquely determined by the conditions under which the experiment is performed.

Characteristics:

• The outcome is always the same for a given set of initial conditions.
• The result can be predicted with 100% certainty before the experiment is conducted.
• These experiments typically follow well-established physical laws or mathematical rules.

Examples:

• Adding two numbers: The result of $5 + 7$ will always be 12. This is a mathematical experiment with a deterministic outcome.
• Boiling Water: Measuring the boiling point of pure water at standard sea-level pressure. The outcome will always be $100^\circ$ Celsius.
• Newton's Laws of Motion: If you drop a ball from a height of 10 meters, the time it takes to hit the ground can be calculated precisely using the formula $s = ut + \frac{1}{2}at^2$. The outcome is predictable.

3. Random Experiment (or Probabilistic/Stochastic Experiment)

A Random Experiment is an experiment where the outcome cannot be predicted with certainty in advance, even if the experiment is repeated under the same conditions. While the specific outcome of a single trial is unknown, the set of all possible outcomes is known.

Characteristics:

• It has more than one possible outcome.
• The outcome of any single trial is uncertain and subject to chance.
• It can be repeated under identical conditions.
• There is a statistical regularity to the outcomes; over a large number of trials, the proportion of times each outcome occurs tends to stabilize.

Examples:

• Tossing a Coin: The possible outcomes are Heads or Tails. We know what can happen, but we don't know what will happen on the next toss.
• Rolling a Die: The possible outcomes are {1, 2, 3, 4, 5, 6}. The result of a single roll is unpredictable.
• Weather Forecasting: Predicting whether it will rain tomorrow is a random experiment, as the outcome is uncertain.
• Drawing a Card: Drawing a card from a well-shuffled deck is a random experiment because any of the 52 cards could be chosen, but which one is uncertain.

The study of probability is concerned exclusively with random experiments.

Outcome and Sample Space:

An Outcome is a single possible result of a random experiment.

The Sample Space of a random experiment is the set of all possible outcomes of the experiment. It represents the complete set of potential results. The sample space is typically denoted by the capital letter $S$ or sometimes $\Omega$. Each individual outcome in the sample space is called a sample point.

The sample space must be defined such that:

• It includes all possible outcomes of the experiment (it is exhaustive).
• No two outcomes can occur simultaneously in a single trial (they are mutually exclusive).

Examples of Sample Spaces:

Let's determine the sample space for various random experiments:

1. Tossing a coin: The only possible results are getting a Head (H) or getting a Tail (T).

   Sample Space, $S = \{H, T\}$

   The number of sample points is $n(S) = 2$.
2. Rolling a standard die: The possible outcomes are the faces showing the numbers 1, 2, 3, 4, 5, or 6.

   Sample Space, $S = \{1, 2, 3, 4, 5, 6\}$

   The number of sample points is $n(S) = 6$.
3. Tossing two coins simultaneously: We need to list all possible ordered pairs of outcomes for the two coins. Let the outcomes be for the first coin and the second coin.
   • First coin H, Second coin H: HH
   • First coin H, Second coin T: HT
   • First coin T, Second coin H: TH
   • First coin T, Second coin T: TT

   Sample Space, $S = \{HH, HT, TH, TT\}$

   The number of sample points is $n(S) = 4$.
4. Rolling two dice simultaneously: We consider the two dice distinguishable (e.g., Die 1 and Die 2). The outcome is an ordered pair $(d_1, d_2)$, where $d_1$ is the result on Die 1 and $d_2$ is the result on Die 2. Each $d_i$ can be any integer from 1 to 6.

   $S = \{(1,1), (1,2), (1,3), (1,4), (1,5), (1,6),$

   $\phantom{S = } (2,1), (2,2), (2,3), (2,4), (2,5), (2,6),$

   $\phantom{S = } (3,1), (3,2), (3,3), (3,4), (3,5), (3,6),$

   $\phantom{S = } (4,1), (4,2), (4,3), (4,4), (4,5), (4,6),$

   $\phantom{S = } (5,1), (5,2), (5,3), (5,4), (5,5), (5,6),$

   $\phantom{S = } (6,1), (6,2), (6,3), (6,4), (6,5), (6,6)\}$

   The number of sample points is $n(S) = 6 \times 6 = 36$. (Using the Multiplication Principle from Counting).
5. Drawing two balls from a bag containing a red (R) and a blue (B) ball, one after the other without replacement: The order of drawing matters.

   Sample Space, $S = \{RB, BR\}$

   $n(S) = 2$.
6. Drawing two balls from a bag containing a red (R) and a blue (B) ball, one after the other with replacement: The order matters, and the possibilities for the second draw depend on the first due to replacement.

   Sample Space, $S = \{RR, RB, BR, BB\}$

   $n(S) = 4$.

Examples of Sample Spaces

Answer:

Answer:

Answer:

Answer:

Answer:

The sample space can be finite (having a finite number of outcomes) or infinite. In introductory probability, we primarily focus on experiments with finite sample spaces. Examples of experiments with infinite sample spaces include counting the number of coin tosses until a head appears (H, TH, TTH, ...) which is countably infinite, or measuring a continuous quantity like time or temperature which can result in uncountably infinite outcomes.

Events

In probability theory, once we have defined a random experiment and its sample space (the set of all possible outcomes), the next crucial concept is that of an event. If the sample space is the "universe" of all possibilities for an experiment, an event is a specific region or a point of interest within that universe. It is the language we use to describe the specific results we want to analyze and calculate probabilities for.

Definition of an Event

An Event is defined as any subset of the sample space S. This simple definition is very powerful. It means that an event is simply a collection of one or more possible outcomes. If the result of our experiment is one of the outcomes included in the event's set, we say that the event has occurred.

Example: Consider the experiment of rolling a single die. The sample space is the set of all possible outcomes, $S = \{1, 2, 3, 4, 5, 6\}$.

• Let's define Event A as "getting an even number". The outcomes that satisfy this description are {2, 4, 6}. So, the event A is the set $A = \{2, 4, 6\}$.
• If you roll the die and it comes up 4, then the outcome '4' is an element of the set A, so we say that Event A has occurred.
• If you roll a 3, the outcome '3' is not an element of the set A, so Event A has not occurred.

Types of Events

Events can be categorized based on the number and nature of outcomes they contain. This classification helps us understand the structure of what we are measuring.

• Impossible Event: An event that can never happen under any circumstances. This corresponds to a subset with no outcomes, which is the empty set ($\phi$). Its probability is always 0.

  Example: For a standard die roll ($S = \{1, 2, 3, 4, 5, 6\}$), the event "getting a 7" is an impossible event. The set of outcomes is { }, or $\phi$.

• Sure Event (or Certain Event): An event that is guaranteed to happen. It consists of all possible outcomes in the sample space. Therefore, the event is the sample space itself ($S$). Its probability is always 1.

  Example: For a die roll, the event "getting a natural number less than 7" includes all outcomes {1, 2, 3, 4, 5, 6}, so it is a sure event.

• Simple Event (or Elementary Event): An event that consists of only a single outcome from the sample space. It is a subset containing just one element.

  Example: For a die roll, the event "getting a 5" is the simple event $\{5\}$. The sample space is the set of all possible simple events.

• Compound Event: Any event that consists of more than one outcome. It can be seen as a combination of two or more simple events.

  Example: For a die roll, the event "getting an odd number" is the compound event $\{1, 3, 5\}$.

Exhaustive and Favourable Number of Cases

These two terms are fundamental to calculating probability in the classical approach.

1. Exhaustive Number of Cases

The total number of all possible outcomes of a random experiment is called the exhaustive number of cases. It is simply the cardinality (or the total number of elements) of the sample space S. It represents every single thing that can possibly happen.

It is denoted by $n(S)$.

Example: In the experiment of rolling a single die, the sample space is $S = \{1, 2, 3, 4, 5, 6\}$. The exhaustive number of cases is $n(S) = 6$.

2. Favourable Number of Cases

The number of outcomes of a random experiment that result in the happening of a particular event is called the favourable number of cases for that event. It is the cardinality (or the total number of elements) of the event set E.

It is denoted by $n(E)$.

Example: Continuing with the die roll experiment, let E be the event 'getting an even number'. The outcomes favourable to E are {2, 4, 6}. Therefore, the set for event E is $E = \{2, 4, 6\}$, and the favourable number of cases is $n(E) = 3$.

These two concepts form the basis of the classical probability formula: $P(E) = \frac{\text{Favourable Number of Cases}}{\text{Exhaustive Number of Cases}} = \frac{n(E)}{n(S)}$.

The Algebra of Events

Because events are sets, we can use the operations of set theory to combine or modify them, creating new events. This "algebra of events" allows us to precisely describe complex scenarios.

Let A and B be two events associated with a sample space S.

1. The 'OR' Event (Union: $A \cup B$)

The event 'A or B' (also written as 'A or B or both' or 'at least one of A or B') represents the set of all outcomes that are either in A, in B, or in both. This corresponds to the union of sets.

$A \cup B = \{x : x \in A \text{ or } x \in B\}$

2. The 'AND' Event (Intersection: $A \cap B$)

The event 'A and B' represents the set of all outcomes that are common to both A and B, meaning they are in A and also in B. This corresponds to the intersection of sets.

$A \cap B = \{x : x \in A \text{ and } x \in B\}$

3. The 'NOT' Event (Complement: $A'$ or $A^c$)

The event 'not A' represents the set of all outcomes in the sample space S that are not in event A. This corresponds to the complement of a set.

$A' = S - A = \{x : x \in S \text{ and } x \notin A\}$

4. The Event 'A but not B' (Difference: $A - B$)

The event 'A but not B' represents the set of all outcomes that are in event A but are not in event B. This corresponds to the difference of sets. It can also be written as $A \cap B'$.

$A - B = \{x : x \in A \text{ and } x \notin B\}$

Relationships Between Events

Mutually Exclusive Events

Two or more events are mutually exclusive (or disjoint) if they cannot happen at the same time. This means they have no outcomes in common. In the language of sets, their intersection is the empty set.

$A \cap B = \phi$

Example: When rolling a die, the event "getting an even number" ($A = \{2, 4, 6\}$) and the event "getting an odd number" ($B = \{1, 3, 5\}$) are mutually exclusive. It is impossible for a single roll to be both even and odd, so $A \cap B = \phi$.

Exhaustive Events

A collection of events ($E_1, E_2, \dots, E_k$) is exhaustive if their union forms the entire sample space S. This means that in any trial of the experiment, at least one of these events is guaranteed to occur. They "exhaust" all possibilities.

$E_1 \cup E_2 \cup \dots \cup E_k = S$

Example: For a die roll, the events "getting a number less than 4" ($A = \{1, 2, 3\}$) and "getting a number greater than 2" ($B = \{3, 4, 5, 6\}$) are exhaustive because their union $A \cup B = \{1, 2, 3, 4, 5, 6\} = S$. Note that these events are not mutually exclusive because they share the outcome '3'.

Mutually Exclusive and Exhaustive Events

A collection of events is both mutually exclusive and exhaustive if they do not overlap and together they cover the entire sample space. They form a partition of the sample space.

Example: For a die roll, the events "getting an even number" ($A = \{2, 4, 6\}$) and "getting an odd number" ($B = \{1, 3, 5\}$) are both mutually exclusive ($A \cap B = \phi$) and exhaustive ($A \cup B = S$).

Answer:

Answer:

Axiomatic Approach to Probability

Early approaches to probability, such as the classical definition ($P(E) = n(E)/n(S)$), were intuitive but had limitations. The classical approach, for example, only works for experiments with a finite number of equally likely outcomes. To create a universally applicable and mathematically consistent theory of probability, the Russian mathematician Andrey Kolmogorov developed the axiomatic approach in the 1930s. This approach defines probability not by a specific formula but by a set of fundamental rules, or axioms, that any valid assignment of probabilities must satisfy. This foundation is so robust that it works for any type of random experiment, whether the outcomes are equally likely or not, and whether the sample space is finite or infinite.

The Three Axioms of Probability

The axiomatic approach begins with a random experiment having a sample space, $S$. Probability is then defined as a function, $P$, that assigns a real number, $P(E)$, to every event $E$ (a subset of $S$). For this function to be a valid probability measure, it must satisfy the following three axioms:

Axiom 1: Non-negativity

The probability of any event must be a non-negative number.

For any event E, $P(E) \ge 0$.

This axiom formalizes our intuition that the chance of something happening cannot be negative. It can be zero (for an impossible event) or positive.

Axiom 2: Normalization

The probability of the entire sample space $S$ (the "sure event," since one of the outcomes in S must occur) is exactly 1.

$P(S) = 1$

This axiom normalizes probability, setting the scale from 0 to 1. A value of 1 represents absolute certainty. The sum of the probabilities of all possible elementary outcomes must equal 1.

Axiom 3: Additivity

If $E_1, E_2, E_3, \dots$ is a sequence of mutually exclusive events (meaning they have no outcomes in common, i.e., $E_i \cap E_j = \emptyset$ for $i \neq j$), then the probability of their union is the sum of their individual probabilities.

$P(E_1 \cup E_2 \cup E_3 \cup \dots) = P(E_1) + P(E_2) + P(E_3) + \dots$

For any two mutually exclusive events, this simplifies to:

If $E_1 \cap E_2 = \emptyset$, then $P(E_1 \cup E_2) = P(E_1) + P(E_2)$.

This axiom is the cornerstone that allows us to calculate the probability of a compound event by breaking it down into simpler, disjoint parts.

Any function $P$ that satisfies these three axioms is considered a valid probability measure, and all other rules of probability can be logically derived from them.

Important Properties Derived from the Axioms

Using only the three axioms, we can prove several fundamental theorems of probability.

• Probability of the Impossible Event: The probability of the empty set (an impossible event) is zero. $P(\emptyset) = 0$.
• Range of Probability: For any event E, its probability must be between 0 and 1, inclusive. $0 \le P(E) \le 1$.
• Probability of the Complement: The probability that an event E does not happen ($E'$) is 1 minus the probability that it does happen. This is an extremely useful rule for "at least one" type problems.

  $P(E') = 1 - P(E)$

  ... (i)

• Addition Rule for any Two Events: The probability of A or B happening is the sum of their probabilities minus the probability of their intersection.

  $P(A \cup B) = P(A) + P(B) - P(A \cap B)$

  ... (ii)

Probability for Equally Likely Outcomes

In many simple experiments, such as tossing a fair coin or rolling a fair die, it is reasonable to assume that every individual outcome in the sample space is equally likely. In this specific but common case, the axioms lead to the classical definition of probability.

For a finite sample space $S$ with $n(S)$ equally likely outcomes, the probability of any event $E$ is the ratio of the number of outcomes favourable to $E$ to the total number of outcomes in the sample space.

Odds in Favour and Odds Against an Event

Sometimes, the likelihood of an event is expressed not as a probability, but as "odds". This is common in betting and gaming.

Odds in Favour

The odds in favour of an event E is the ratio of the number of outcomes favourable to E to the number of outcomes unfavourable to E.

Odds in Favour of E = $\frac{\text{Favourable Outcomes}}{\text{Unfavourable Outcomes}} = \frac{n(E)}{n(E')} = \frac{n(E)}{n(S) - n(E)}$

If $P(E)$ is the probability of the event, then the odds in favour are $\frac{P(E)}{1 - P(E)}$.

Odds Against

The odds against an event E is the ratio of the number of outcomes unfavourable to E to the number of outcomes favourable to E.

Odds Against E = $\frac{\text{Unfavourable Outcomes}}{\text{Favourable Outcomes}} = \frac{n(E')}{n(E)} = \frac{n(S) - n(E)}{n(E)}$

If $P(E)$ is the probability of the event, then the odds against are $\frac{1 - P(E)}{P(E)}$.

Answer:

Answer:

Answer:

Answer:

Laws of Probability

The three axioms of probability provide the fundamental foundation of the theory. From these axioms, we can derive several powerful laws that allow us to calculate the probabilities of more complex, combined events. These laws, including the Addition and Multiplication Rules, are the primary tools used in solving probability problems.

Important Properties and Laws Derived from the Axioms

Using only the three axioms, we can prove several fundamental theorems of probability.

• Probability of the Impossible Event: The probability of the empty set (an impossible event) is zero. $P(\emptyset) = 0$.
• Range of Probability: For any event E, its probability must be between 0 and 1, inclusive. $0 \le P(E) \le 1$.
• Probability of the Complement: The probability that an event E does not happen ($E'$) is 1 minus the probability that it does happen.

  $P(E') = 1 - P(E)$

  ... (i)

Law for Subsets

If event A is a subset of event B (meaning every outcome in A is also in B), then the probability of A is less than or equal to the probability of B.

Intuitive Explanation: When we add $P(A)$ and $P(B)$, we have double-counted the probability of the overlapping region (the intersection, $A \cap B$). To correct this, we must subtract the probability of this intersection once.

Derivation

Since $A \subseteq B$, we can express event B as the union of two mutually exclusive events: A and the part of B that is not in A (which is $B - A$).

$B = A \cup (B - A)$

Because A and $(B-A)$ are mutually exclusive, we can apply Axiom 3:

$P(B) = P(A) + P(B - A)$

By Axiom 1 (Non-negativity), we know that the probability of any event must be greater than or equal to zero. Therefore, $P(B - A) \ge 0$.

This implies that $P(B) \ge P(A)$, or $P(A) \le P(B)$.

The Addition Rule of Probability

The Addition Rule (also known as the Sum Rule) is used to find the probability of a union of events. It answers the question: "What is the probability that at least one of a set of events will occur?"

1. Addition Rule for Any Two Events

For any two events A and B in a sample space S, the probability that either event A or event B (or both) will occur is given by the formula:

Derivation and Intuition

Imagine calculating the probability of the event $A \cup B$. From set theory, we know that we can partition the union into three mutually exclusive parts: $A - B$, $B - A$, and $A \cap B$.

So, $A \cup B = (A - B) \cup (B - A) \cup (A \cap B)$. Since these three events are mutually exclusive, by Axiom 3:

$P(A \cup B) = P(A - B) + P(B - A) + P(A \cap B)$

We also know that $A = (A - B) \cup (A \cap B)$. These are mutually exclusive, so $P(A) = P(A - B) + P(A \cap B)$, which gives $P(A - B) = P(A) - P(A \cap B)$.

Similarly, $P(B - A) = P(B) - P(A \cap B)$.

Substituting these back into our equation for $P(A \cup B)$:

$P(A \cup B) = (P(A) - P(A \cap B)) + (P(B) - P(A \cap B)) + P(A \cap B)$

$P(A \cup B) = P(A) + P(B) - P(A \cap B)$

The intuition is simple: when we add $P(A)$ and $P(B)$, we have counted the probability of the intersection, $P(A \cap B)$, twice. Therefore, we must subtract it once to get the correct total probability.

2. Addition Rule for Mutually Exclusive Events

When two events, A and B, are mutually exclusive, they cannot happen at the same time. This means their intersection is the empty set ($A \cap B = \emptyset$), and consequently, the probability of their intersection is zero, $P(A \cap B) = 0$. In this special case, the Addition Rule simplifies to:

This is a direct application of the third axiom of probability.

3. Addition Rule for Three Events

The principle can be extended to three events A, B, and C. This is an application of the Principle of Inclusion-Exclusion.

Intuitive Explanation: To find the total probability (area) covered by the three events:

1. We first add the probabilities of the three individual events: $P(A) + P(B) + P(C)$.
2. In doing so, we have double-counted the regions where two events overlap ($A \cap B$, $B \cap C$, and $A \cap C$). So, we must subtract their probabilities once.
3. However, when we subtracted the three pairwise overlaps, we subtracted the central region where all three events overlap ($A \cap B \cap C$) three times. Since it was initially added three times (once with each event), it has now been completely removed. We must add it back once to get the correct total.

Conditional Probability

Often, the probability of an event changes if we know that another event has already happened. Conditional probability is the probability of an event occurring, given that another event is already known to have occurred.

The conditional probability of event E happening, given that event F has already happened, is written as $P(E|F)$.

Intuitive Explanation: When we know that event F has occurred, our world of possibilities shrinks. The original sample space $S$ is no longer relevant; our new, reduced sample space is now just the event F. The favorable outcomes for E are now only those outcomes that are also inside this new sample space F, which is the intersection $E \cap F$.

The formula is:

The Multiplication Rule of Probability

The Multiplication Rule is used to find the probability of an intersection of events. It answers the question: "What is the probability that both event E and event F will occur?" This rule is derived directly by rearranging the formula for conditional probability.

In words: The probability of both E and F happening is the probability of E happening, multiplied by the probability of F happening *given that E has already happened*.

Multiplication Rule for Independent Events

Two events are independent if the occurrence of one does not affect the probability of the other. For example, tossing a coin and rolling a die are independent events. If E and F are independent, then knowing F has occurred doesn't change the probability of E, so $P(E|F) = P(E)$.

In this special case, the Multiplication Rule simplifies to:

Crucial Distinction: Do not confuse "mutually exclusive" with "independent." They are nearly opposite concepts for events with non-zero probability.

• Mutually Exclusive: If one happens, the other cannot. ($P(A \cap B) = 0$)
• Independent: If one happens, it does not change the probability of the other. ($P(A \cap B) = P(A) \cdot P(B)$)

Law of Total Probability

This law allows us to find the probability of an event A by considering a set of mutually exclusive and exhaustive events that partition the sample space. Let $E_1, E_2, \dots, E_n$ be events that form a partition of the sample space S (i.e., they are mutually exclusive and their union is S). Then the probability of any event A can be expressed as:

Answer:

Answer:

Answer:

Answer: